On the scalability of Big Data Cyber Security Analytics systems

Big Data Cyber Security Analytics (BDCA) systems use big data technologies (e.g., Apache Spark) to collect, store, and analyse a large volume of security event for detecting cyber-attacks. The digital in general specific is increasing exponentially. velocity with which generated fed into BDCA system unpredictable. Therefore, should be highly scalable deal the unpredictable increase/decrease data. However, there has been little effort investigate scalability identify exploit sources improvement. In this paper, we first Spark-based default Spark settings. We then configuration parameters execution memory) that can significantly impact system. Based on identified parameters, finally propose parameter-driven adaptation approach, SCALER, optimizing system's scalability. have conducted set experiments by implementing large-scale OpenStack cluster. ran our four datasets. found (i) settings deviates from ideal 59.5% (ii) 9 out 11 studied (iii) SCALER improves 20.8% compared parameter setting. findings study highlight importance exploring space underlying framework cyber analytics.